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SYSTEM FOR AUTONfATED GENERATION method provides for selecting some of those media elements 

OF MEDIA io response to a request for media programming, and select- 
ing a temporal organization for the media elements. 

RELATED APPLICATIONS However, the temporal organization is not dictated by the 

■ iron i5 sclcctcd information that regarding cach of thc mcdia clc- 

This application clamis pnority from U.S. Provisional .-ru _ i-* j-5 .w— ^. 

n«»^-» A i-,,.*-^- xT« /in rvTo fii-^ i ioot .„K;«k mcnts. The svstem sele<^ and orders the media elements 

PatentAppucationNo. 60/042^64, filed Apr. 1,1997, which » • . j . • .i_ * j ^ ' . • 

. . i^*. , according to the data in the request, and accordmg to 

is hereby mcorporated by reference in its entirety. . - . . ' 

' *^ ' ' information, such as permitted transitions, regarding the 

FIEKD OF THE INVHN'llON media elements. The system prevents a user from selecting 

10 individual media elements. The media elements are then 

This invention relates to a method and computer- assembled into media programming, 

implemented system for creation of audiovisual program- ^^^^^ the invention, a method is provided 

for verifying viev^ong and comprehension of a unique media 

BACKGROUND OF THE II^NTION „ P'«8.f»'"- -"'""^ ''^l"''* providing, in the uniq^ 

1^ media program, a umque sequence of cues. The method 

Hiere have been recent substantia] advances in the capac- includes receiving from a viewer of the unique media 

ity to design customized audio visual programs for specific program information relative to said cues, such as responses 

purposes from a library of existing video clips and audio to questions included on a questionnaire, or in response to a 

elements. Customization of audiovisual programing is use- telephone call made by the viewer. The received information 

ful in many applicatioas. For example, in advertising certain 20 is then compared to the sequence of cues to determine 

products, and in particular automobiles, one promotional whether or not the program was viewed, and the level of 

technique is to prepare promotional videotapes which are comprehension by the viewer. 

sent to potential customers on their request. TTie desirability in another aspect of the invention, a method of aeating 

of customizing such videotapes to demographic or other audiovisual programming from stored audiovisual media 

characteristics of individual consumers arc of course sub- 25 elements is provided. In a first step, from a database con- 

l^j stantial. Health care practitioners and managed care entities taining information concerning the audiovisual media 

have begun to provide instmctional videotapes to patients elements, certain audiovisual media elements are selected. A 

S| with inforrnation regarding managing of various diseases temporal sequence for the selected elements is designated. 

!p and conditions. Customizing of such information to the Transitions between the media elements are automatically 

rsa* disease and condition of the individual, and demographic 30 selected. 
W characteristics of the individual, such as age, income, edu- 

y cational level, psychographic characteristics such as per- BRIEF DESCRIPTION OF THE HGURES 

ceived weUness and willingness to change behaviors, and FIG. lis a schematic diagram of a system according to the 

•sa. other factors, would be valuable for increasing the effec- invention. 

3 tiveness of such video tapes in communicating the informa- 35 piG. 2 is a schematic diagram indicating information 

ITl tion to the reapient. contained in a database according to the invention. 

R| ^° accordance with present technology, it is possible to piQ 3 ^ schematic diagram indicating the organization 

.>jt create and store a Ubrary of brief video clips, and provide a 3 database according to the invention. 

fif database of information regarding these cUpsMIowever. in piG. 4 is a schematic diagram indicating software cle- 

1^ accordance with the present technology, a hiunan editor ^ ^^^^ ^^^^^^ ^^^^^.^^ 

1^1 must make the ultimate selection of individual cups, and ^ . / , . . , , . ^ 

^ provide the final editing decisions, creation and selection of ^ ^ flow chart showing steps m the assembly of an 

transitions so that there is a smooth visual and audio "^^^^^^ accordmg to the invenUon. 

transition between adjoining chps in the program, and FIG- ^ »s a flow chart showing steps in the assembly of 

checking of the content of the clips to determine that there programming in a method according to the invention, 

is proper coverage of the appropriate subject matter in an FIG. 7 is a flow chart showing steps in a method according 

appropriate sequence. Automating of this editing process to the invention of verifying viewing of a program, 

would make possible substantial flexibility and new possi- DETAILED DESCRIPTION OF A PREFERRED 

biliiies for creation of audiovisual programming. EMBODIMENT 

Once videotapes have been provided to the user, it is n r ■ * r^r^ 1 *l - ^ • ♦ j « -ia j 

A'«z 1* # A u *u .41. L • A *u Referrmg to FIG. 1, there is depicted a system 10 accord- 

difflcult to vcniy whether or not the user has viewed the * • *• e * in • 1 j * 

r- f .u u u • J *u 1 1 c mg to the mvention. System 10 mcludes a computer 20 

program. Even if the program has been viewed, the level of .j- An jj i 

t ' ' u* mcludmg processor 30, mput devices 40, output and display 

comprehension is difficult to assess. , . f n , 1 • icn ^ u 

devices 50 and memory devices 60. Computer 20 may be a 

h is accordingly an advantage of this invention that the 55 conventional personal computer with suitable processor 
disadvantages of the prior art may be overcome. ^^^^^ memory capacity and speed. Computer 20 corn- 
Additional advantages of the invention, and objects of the municates in a suitable maimer with Ubrary 70 of audiovi- 
invention, will become apparent from the detailed descrip- sual clips 80. Library 70 may be stored on any suitable 
tion of a preferred embodiment which follows. storage medium of adequate bandwidth and retrieval time 
01 T\/Kx ADv nx: xun TKn/cKmrkKT ^ characteristics. At present, certain fixed or bard disk systems 
SUMMARY OF THE INVENTION ^^^^-^^ ^^p^^^^ performance to other options. Digitized 
According to a first aspect of the invention, a system and audiovisual clips 80 are stored in library 70. Each audio vi- 
method of creating media programming are provided. A sual clip is suitably identified, such ashy the name of one or 
database is provided which contains selected information more files which make up the clip, 
about each of a large number of media elements. The media 65 Database. 

elements may be, for example, audiovisual clips, llie ele- Computer 20 is also suitably associated with a database 

ments themselves are maintained in a suitable library. The 100. Database 100 contains unique identifying information 
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for each clip and has associated therewith additional infor- family of defining values which describe the range of the 

mation often arranged in a hierarchical manner. Referring to target audience in specific demographic and psychographic 

FIG. 2, this information will be referred to herein as a header terms. Additionally, the overall outline of the subject matter 

no. Individual items of information within the header will is contained in this layer and is associated with demography 

be referred to herein as tags. The tags arc divided into two s where appropriate. These outlining fimctions arc secondary, 

general categories, namely control tags 115 and content tags however, to the temporal organizational templates embodied 

120. Content tags 120 contain information identifying the in this layer. Here the instructional designer, or interactive 

content of the clip. The content is often embodied in three author, defines the preferred temporal modes of presentation 

general categories, namely content, i.e., information com- of the universe of assets. For example, the instructional 

municaled by the clip, as indicated by block 125, viewer lo designer might define that the block of programming content 

profiles for viewers most likely to be receptive to the clip, called EFFECTS ON THE HEART is presented across three 

indicated by block 130, and intended use of the clip, fundamental age groups, two levels of detail (summary/ 

indicated by block 135. The a)ntenl tags contain information cursor)' and in-depth), both gender specific groups and four 

defined by a suitable system of classification of information. distinct ethnicity components. Within this multi- 

For example, in a database of clips for the assembly of video 15 dimensional array of program assets, the instructional 

programs concerning medical information, the content infor- designer might also define that the material be presented in 

mation may, in addition to other systems, such as for the preferred sequence of — INTRODUCTION TO THE 

categories of content mentioned above, make use of Inter- HEART, IMPACT OF DIABETES ON THE 

national Classification of Disease codes. In databases CARDIOVASCULAR- VASCULAR STRUCTURES, 

directed to assembly of programs for other topics, other 20 EFFECTS OF DIET, EFFECTS OF EXEROSE, O&A 

jffiSi coding systems are used for the content information. Fields SESSION, SUMMARY. 

Q may be provided to indicate technical complexity, specific Below the instructional design layer are the smaller 
items of information conveyed, demographic characteristics organizational elements which allow for elasticity in the 
Q of actors in clips, style of script, and other suitable Infbr- specifics of the implementation of the temporal design. This 
mation. One or more fields may be provided within the 25 is called the MODULE LAYER 320 and in special instances 
ty content lag tu define content. The system by which the the SEQUENCE LAYER. Fundamental lu this layer are 
content information is defined is preferably organized to weighting factors which control likelihood of asset use, and 
W provide for relatioaships among types of information. Such allow for the deployment of elements which are free to float 
relationships may be levels of generality, and other subject temporally in order to accomplish certain transitions and 
matter relationships. 3U effective deployment of those elements which are slave to 
The organizational structiirc of the database may be the temporality functions. These elements as a group are 
M hierarchical, with each layer of hierarchy defining a specific shorter sequentially-patterned program elements of content 
IP set of organizational principles. Referring to FIG. 3, at the which organize under the temporality principles of the 
[%| highest level the global behavior of the universe of elements higher layer. The free floating elements may have various 
is defined. This is equivalent to automating a 'look-and-feel* 35 linking geometries or parameters at the opening and closing 
I y function for the entire unit of programming. A given stylistic thereof. Such elements can be used to bridge elements that 
1^ approach 300 can be defined through the use of default caunot themselves be linked because a direct link is either 
|<ss| values which will be employed, for example, in the treat- disallowed or would involve use of disfavored transitions. 
!^ menl of transitions between adjacent clips. There may be The lowesl level of organization is that of the individual 
\ l| defined a set or range of transition values, such as a range 40 media elements or assets themselves, lliis is called the CLIP 
from hard-aJt to 24-frame dissolve. In this example, the LAYER 325. These elements carry tags which define Iheir 
master look-and-feel level of organization might define specific content, such as: DIABETIC HEART, LEFT 
hard-cut as the aesthetically optimal, and therefore the VENTRICLE, DAMAGE TO, HYPERGLYCEMIA, 
default, choice. Numerous other aesthetic or programmatic MALE, AGE 50, TALKING HEAD. The first three content 
choices that affect the delivery and placement of media 45 tags will be noted as being in hierarchical order from most 
elements in the concatenated stream can be defined. general to most specific. The next two are examples of 
Also at the highest level of organization, typically used in demographic tags, and the final tag is a simple example of 
on-line applications only, there may be provided the viewer/ a tag denoting style. These elements also carry production- 
user interface options which define the ways in which any specific control tags, which, as discussed in more detail 
given class and security level of user will be allowed to so below, define such characteristics as allowable exit/entrance 
actively as well as passively interact with media assets. We transitions for both audio and video, 
will call this the INTERFACE LAYER 305, At this level of It is important to note that, in the operation of the system, 
organization, the behaviors of ancillary assets such as pro- there is an inhibition layer between the clip and the search- 
motional segmenLs information identifying the system, ing mechanism. The inhibition layer a.ssures that the system 
advertisements and news-flashes are defined. These assets 55 does not include in the programming every clip that is 
embody aesthetic, program or instructional design, as well responsive to a query provided by a user. The inhibition 
as market-driven, or viewer defined behaviors. mechanism may be responsive to psychographic character- 
Immediately below this layer is preferably the meta- istics of the user, such as age, level of education, or even 
content layer. This is called the PROGRAM LAYER 310. reason for the query. The tags arc responsive to this type of 
Here are defined the type of assets and the core content 60 information. The inhibition mechanism may be modified 
descriptions of those assets. By way of example, the types of dynamically as the database is mined for suitable assets. The 
assets may be defined as training, informational, and enter- inhibition mechanism may be viewed as in a multi- 
taioment assets. Examples of core subject matter would be dimensional set of psychographic requirements. Clips can be 
"medical", at the highest level, "health management", at a evaluated for their responsiveness in the various dimensions, 
lower level, and "diabetes mellitus", at a still lower level. 65 The system may set, or the user may select, a length of time 
Next in the hierarchy is the instructional design layer, or for the program, and the inhibition mechanism will operate 
TEMPI ,ATE I AVHER 315. This layer is characterized by a to limit the total number of clips selected to the selected 
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time, as well as choosing clips accordiog to suitability for 
the viewer and the viewer's purpose. 

Referring again to FIG. 2, control tags define audio and 
video components of clips. The audio and video information 
mav be entirely separate as indicated by block 140, VIDEO 5 
CONTROL TAGS, and block 150, AUDIO CONTROL 
TAGS. For example, a minimimi number of control tags 
would be, for each of audio and video, transition points 145, 
155 an optimum transition point from the beginning of the 
clip, a first possible transition point from the beginning of 10 
the clip, a last possible transition point firom the beginning 
of the clip, an optimum traasitioo point from the end of the 
clip, a last possible transition point from the end of the clip, 
and a first possible transition point from the end of the clip. 
Each of these points may be defined in time, by video 15 
frames, by tags of cfips ihai may be the subject of the 
transition, or by content parameters. These points may be 
defined by human post-production editors entering such 
information into the database in response to suitable prompts 
contained in the software. These points may also be defined 20 
by truth-table or expert -system defaults, which may or may 
not reference either video or audio content The variable 
beginning and exit points may be referred to as creating 
fiizzy extremities on the clips. This is beheved to be a novel 
feature as compared to existing databases of audiovisual 25 
assets, and lends significant flexibility to the process of 
creation of transitions. Numerous optional control tags can 
be used, such as, with reference to video information, a 
luminance range 160 for the opening and closing of tbe clip, 
preferred or required transitions^ including length and type 30 
of transition, a dominant chroma value for each of the 
opening and closing of the clip, stan and cod points for 
dialog and action. With reference to audio information, 
control tags may include key of music 170, tempo of music 
175, preferred transitions and other information. This type of 35 
information can be derived by suitable software reviewing 
tbe data stream before or after tbe clips are concatenated, or 
in advance of the assembly process using clip analogs or 
proxies containing suitable information from tbe tags. 
Creation of Database. 40 

The database is created by identifying each chp or other 
asset and defining values of the control tags and content tags 
for each. Values of the various control tags and content tags 
may be defined by a user, either during development of the 
script for the clip or upon completion of the clip. For 45 
example, the program may include screens prompting a user 
to select a value for each control tag from a menu of options. 
Different screens may be provided for different users, such 
as dialog editors, audio editors, and video editors, to enter 
values for appropriate control and content tags. 50 
Alternatively, values of various tags may be created auto- 
matically by truth tables or decision-capture systems, or 
other automated techniques, either with or without human 
assistance. Such systems may act from intbrmatioo derived 
from analysis of existing clips using image-recognition 55 
software, from analysis of scripts entered into text, or from 
other information provided by hiunan editors. 

By way of example only, a system for creation of a 
database for use in connection with creation of medical 
programming will now be described with reference to FIG. ^ 
4. The system may include five primary components, each of 
which will be provided with its own user interface: 

1. A production interface 405 which provides an interac- 
tive means of importing a video chp into the system and 
recording the transition production information. 65 

2. A coding interface 410 to allow the entry of the medical 
and patient selection codes which apply to the clip. 
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3. A patient profile entry interface 415 and decision list 
processor. 

4. A decision/transition list editor 430 with buUt-in tran- 
silitin previewer for both audio and video compcmenLs 
of ctips. 

5. A production player/recorder which embodies an 
engine for both audio and video transitions. 

Each of the foregoing components operates as an inde- 
pendent entity in the exemplary system. Each component is 
started by the user selecting an icon from an appropriate 
folder. The user interface is preferably be graphical in 
nature. Actual keyed data entry is preferably kept to a 
minimum where most entries are made by selecting an 
option from a set or an entry from a list. 

The Production Interface 

The production interface 405 is the interface designed to 
provide a stmcmred, interactive means of importing a video 
clip into the system. A standard container, such as OMF, may 
be utilized as a standardized vehicle for the transmission of 
the proprietary tags. When selected, the user will be pre- 
sented with a screen containing fields for the entry of the 
following: 

Qip ID — ^TMs will become the primary key or file name. 

Source — ^Where the clip will be read from. Options may 
include DAJ' tape or network. 

The user will also be presented with one or more inter- 
faces to enter the audio/video transition fields and the coding 
(medical^atient demographic) information. Before being 
allowed to exit the process, defaults will be applied to the 
transition fields not specified by the user. The user may be 
required to preview the clip just imported. 

The Coding Interface 

The coding interface 440 is the GUI designed to perform 
the entry of the medical and socio-demographic selection 
codes which apply to the cHp. These include the ICDs for 
which this clip is applicable, the socio-demographic and 
medical attributes of those patients assumed to the most 
likely potential viewers of the clip, and any special fields 
which when present in the client/patient pn)rile will cau.se 
this chp to be selected. All fields may be selected from 
preestablLshed lists or groups displayed as ''radio-buttons". 
As the ICD set may require the designation of hundreds or 
thousands of codes, a suitable method of use of the hierar- 
chical nature of the ICD structure may be utilized to simplify 
the selection process. These may be selected during the 
script outlining or scripting process by the use of menus or 
existing fields incorporated in the scripting software. 

The Patient Profile Entry Interface 

The Patient profile entry interface 415 is the GUI 
designed to perform the entry of the general, medical and 
socio-demographic codes for the specific patient for which 
the product is to be produced. Tbe general section may 
require the most data entry including patient name, address, 
.social security number, insurance billing numbers, referring 
entity identification and any text which is to be added to the 
final video. Systems, such as those used in the health care 
industry, may be employed to extract relevant information 
firom existing patient records. In the case of on-line use, the 
medical and socio-demographic sections will be "click to 
select" entry similar in namre to the entry process performed 
by the coder when the clip is imported. The result of this 
process will be to create an entry in the patient database 420. 
Tbe information is then forwarded to the editor program 
425. 

Based on the topic requested, such as a condition which 
the patient has been diagnosed with, and the demographic 
information, editor program 425 will, as discussed below. 
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produce a recommended dedsioD list, or preliminary edit Editor Program. 

decision list (EDL) file for further processing. Subsequently, Programs are created from clips by an editor program, 

the final EDL is created in an evolutionary manner. At first. The identify and order of the clips that comprise a 

a sample order of clips, without transitions, is analyzed. The program may be defined by a variety of methods varying in 

order and identities of the clips is then revised. When the s the degree of control exercised by the individual user. At the 

order and identities of the clips has been finalized, the level of least control by the user, the user defines only the 

transilioas are uimpuled. The transitions are then executed overall functional requiremenLs for the program. For 

and inserted into the stream by use of a system of removal example, in the assembly of programs for education of 

of portions of the tag extremities used in temporal transi- patients about health issues, the user may define only the 

lions. demographic characteristics of the intended recipient of the 

The List Editor/Prcvicwcr program and the information to be conveyed, e.g., managc- 

Thc List Editor/Pre viewer 430 is the GUT designed to ment of a particular condition. In one embodiment, the 

provide the production editor with the abOity to change, add system may simply select appropriate clips from the data- 

to and preview the EDL that has been produced by the editor base and apply suitable transitions. Alternatively, an expert 

program 425 in response to information entered in the system included within the editor program selects and orders 

Patient Profile Entry Intcr&cc 415. one or more suitable templates as shown in block 505 of 

The Production Player FIG. 5. The template may incorporate the demographic 

Each clip, clip component, or other media asset, is stored characteristics and information to be conveyed, or may 

in three distinct segments: contain only sequential requirements for the underlying 

1. lead-in segment structures used in the program. Using the requirements in the 

2. main body 20 selected templates and the demographic characteristics and 
3 gxij segment ^^'^^ requirements specific to the program, as well as the 
Transitions can only be performed on the lead-in and out inherent structural characteristics of the modules and 

segments. The production player 435 is launched with the sequences, the expert sj-stem or truth table then selects and 

name of the patient EDL to be played. Tht EDL will be orders one or more suitable modules and floater sequences, 

analyzed and each transition rendition time estimated. The 25 as shown by block 510 of FIG. 5. Using the requirements in 

production player will then process each command in the the selected modules and the inherent structural character- 

EDL. Most commands, which require very little processing, istics of sequences, the expert system or truth table selects 

cause a more detailed command to be written to the player and orders suitable sequences, as shown by block 515 of 

FIFO which controls the player output thread. Other process FIG. 5. Finally, using the requirements in the selected 

intensive commands, such as a transition, will be rendered, 3U sequences and the inherent structural characteristics of the 

stored to disc and the detailed command with its disc clips, the expert system or truth tabic selects and orders 

location is added to the player FIFO. As each EDL command suitable clips, as shown by block 520 of FIG. 5. The expert 

is processed, the remaining estimated time flow is modified system or truth table also makes transition selections at the 

until the program determines that future transition rendering beginning and end of the audio and video portions of each 

will be able to be completed before the player output thread 35 clip, as shown by block 525 of FIG. 5. At any level in this 

requires it. At this point, the player output thread will be process, if there is an incompatibility, the expert system or 

started taking its input iustructioos from the player FIFO. truth table may discard an element aod replace it with a 

The player output thread will operate asynchronously, trans- substitute element, or employ suitable transition methods, 

fcrring each data uimponeni specified in the player FIFO to For example, if it is not possible to assemble acceptable 

the appropriate hardware component, such as video and 40 sequences in the manner dictated by the modules under the 

audio decoder modules, text merge processor, etc., whose template, the expert system or truth table may discard one or 

output drives the device for recording the program, such as more modules and select alternatives that meet the require- 

a videotape deck. ments of the template. 

Upon completion, the required managerial and billing The tags may include information regarding such 

information will be generated and stored in an appropriate 45 attributes as luminance, chrominance, music tempo and key, 

database. and colors, at the beginning and end of each clip. 

The transition process will require the identification of Alternatively, a MIDI record of the audio portion of the clip 

both an exit and lead-in point, and a transition fiinction may be analyzed. The editor program may apply expert 

specification. This pair of fields are refined into a single system software or truth tables to determine whether a direct 

recommended field as defined above, and then the final 50 transition between any two dips meets aesthetic require- 

recommendedEDLis obtained. These fields are contained in ments. If not, the system may identify a suitable type of 

the preliminary EDL which is being played. The appropriate transition for both clips, add a bridge between the clips, or 

time positions are located in both the exit and lead-in determine that one of the clips must be discarded. For 

segments. The first Tnime of each segment is decoded into example, if the keys of the music in two adjacent clips are 

separate buffers and then the transition function is applied to 55 incompatible, the system may add a bridge consisting of a 

each matching pixel of bo\h images to produce the third burst of percussion in the audio portion of the program, 

rendered frame buffer. A similar process is also appUed to The viewer or user may directly select templates, 

the matching audio sample points associated with the cur- modules, or sequences, depending on the degree of control 

rcnily rendered frame. The video and audio is then desired. Decisions regarding cither specific production dcci- 

re-compressed and written to the output transition file. 60 sions or global stylistic choices or content or order can be 

Values of certain control and content tags, such as key or captured by the system by, in addition to expert systems, 

tempo of music, may be determined by suitable software truth-table or other decision tree programming specific deci- 

from the clip. sion coUection fields provided to content developers. Expert 

It is to be emphasized that the foregoing system is merely systems, truth tables and other systems may be used to create 

exemplary, and does not limit the scope of systems that may 65 the lags associated with the cHps. 

be implemented within the scope of the inventions described Use of expert systems, or decision-capture systems is of 

in this application. interest because the system might organically evolve .siylis- 
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tic tendencies which might mimic or minor those of creative can still be employed to generate useable dissolves. Using 

professionals. Templates might be provided to content pro- this technique, the full decode process can be further 

viders or even to end-usei5, which would allow a specific reduced, and thus accelerated, by processing this luminance 

style of editing, audio cutting or mixing, or program foraia- in averaged blocks of pixels. These pixel blocks are 

tion; perhaps a template may be provided associated with an 5 employed in the encode process. Even without the use of 

individual editor. Such decision-capture systems already MPEG, it is possible that some or all of these shortcuts might 

exist for other uses which could be adapted to the assembly be effectively employed to accelerate the creation of dis- 

of audio and video. solves without use of a full decode/recode cycle. 

In one embodiment, the user creates queries depending on Viewer Database, 

the qualities desired for the organization levels to be iden< lO Separately from the database of media assets described 

ti6ed. The editor program then identifies suitable templates, above, there may be provided a database of viewer iofor- 

modules or sequences based on the queries. The relative mation. This database defines the identity of each viewer, 

importance of dirferenl ilem.s in the queries may be including name and address informalion, and may also 

weighted. The weighting may be supplied by the user in include social, economic and medical data regarding the 

response to suitable prompts, or may be generated automati- 15 viewer, the history of items viewed by the viewer, and a file 

cally by the system based on existing programming. Using of preference information for the viewer. In an on-line 

suitable relationships among data items, the user may be environment, the viewer may be prompted to provide viewer 

presented by the editor program with one or more templates, identification information which may be entered into a 

modules or sequences that represent a fundamental database or may be retained temporarily as a viewer data 

response, another set that represent a secondary response, a 20 record for use in creation of customized programming, 

third set that represent a tertiary response, and so forth. Verification of Viewing. 

Assembly Program. The verification of viewing of the delivered video pro- 

Once the set of clips has been defined, the assembly of the gram is difficult to accomplish. It is important for the 
clips from the library takes place. This is accomplished by well-being of the viewer when the program contains thera- 
an assembly program. Referring to FIG. 6, the assembly 25 peutic or training video programming, and useful for all 
program reads a file of clip identifiers and transition infor- content to assess the effectiveness of delivering infoimatiun 
mation aeated by the editor program as shown by block via customized video programming. An effective system 
605, and calls to the library for each identified clip as shown would: (a) allow the distributor of the programming to know 
by block 610. The identified clips are then transmitted from if no effort had been made on the part of recipient to watch 
the library to a suitable destination. The assembly program 30 the delivered material; (b) provide disincentives for fast- 
uses the transition information in assembling the clips into forwarding and skipping through the program; (c) allow for 
the program. If the clips are transmitted to a suitable digital the confidential verification of the particular program 
to analog converter, the clips may be assembled into a watched, thus allow for the confidential generation of 
program on videotape without compression. However, in viewer-comprehension testing; (d) provide incentives, per- 
order to assemble the program as a digital audiovisual 35 haps monetary or material, for the successful viewing of the 
program in memory, or to transmit the program over tele- material; (e) maintain confidentiality of the viewer and/or of 
phone lines or the Internet, data compiessioo is required. the precise content of the delivered program. The following 
Suitable data compression techniques and standards may, be viewing verification system performs all of the objectives 
u.sed. outlined above. 

Ihe assembly program may also dictate particular tran- 40 Referring to HG. 7, the program software creates an 

sitions. For example, asymmetrical transitions in audio are identifier sequence of cues, as indicated by block 705. The 

advantageous. A leading edge audio transition which is cues may be numbers, letters or other indicia, such as colors, 

exponential and short, and a trailing edge transition in audio shapes, sounds or combinations thereof, unique to each 

which is linear and long is preferred. The video transitions viewer and each program. The sequence may be derived 

need not match the audio transitions. 45 from such information as the viewer's ID number, such as 

Hie following is a technique which is believed by the the Social Security number or other unique record locator 

inventor to be suitable for improving performance of com- contained in the patient/viewer record database, the unique 

pression of the programming, for use in compression segment and/or program ID numbers or the content of clips 

formats, of which MPEG is an example, which achieve or other portions of programming delivered to the viewer, 

performance using predictive compression wtiich includes 50 Regardless of the method employed, a unique key sequence, 

both forward and backward looking frames. The technique or unique sequence of cues, which may be alphanumeric or 

is the elimination of the complex frames from the MPEG comprise a series* of images, icons, colors or sonic cues, is 

stream using the last-in and first-out video marker tags to created, 'lliis sequence is imique to the customized video 

determine the amount of leading and trailing video which program. The elements of the sequence are recorded within 

might, in a worst-case scenario, participate in a visual effect. 55 the program, as indicated by block 710. The cue are pref- 

By eliminating these (P and B) frames, it is possible to erably spread throughout the temporal sequence of the 

employ a partial decoding of the (Hufi&nan) algorithm, rather viewing experience one or more cues at a time. By way of 

than a full decode/encode cycle. Additionally, the elimina- example, numbers in the sequence may l>e superimposed 

tion of these PB frames allows impunity in the concatenation over picture elements using character generation and keying, 

points employed. This freedom is bought at the price of 60 The picture sequences which underlie these characters may, 

increased bandwidth requirements, but only for small scg- or may not address the superimposed characters/numerals, 

ments of video rather than entire clips. In the case of audio cues, it is of course more likely that a 

Another technique is applicable to compression formats, specific sequence, such as a talking head, would be 

such as MPEG, where the chrominance data is subsidiary to employed to communicate the cue. A special case of the cue 

the luminance data. This technique involves emulating the 65 is the use of actual questions, usually relating to viewer 

MPEG transitions by extracting luminance-only data. comprehension of the presented material and having mul- 

Becau.se the chrominance data 'rides on* the luminance, this tiple choice responses. 
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As indicated by block 715, the viewer is provided with a is, at tninimiim, the verification of active viewing of the 

method for response. The response method includes suitable temporal stream of information. The responses are com- 

method to record these numbers, characters, colors, shapes pared to cues and expected responses, as indicated by block 

or other cues contained within the sequence. For viewers 725. At maximum, the information is also capable of veri- 

who arc receiving the programming online through a modem 5 fying time of viewing, comprehension of content, attitudes 

or Internet link, this might be a window which remains open regarding content (by asking for reactions in some or all of 

and active during the playing of the video sequence, or the fields), as well as motivating vieweis vdih the possibility 

which becomes open and active upon the presentation of the of prizes, secrets, entry into further lotteries or drawings, or 

appropriate superimposed strings, which allows the viewer other perceived rewards. The information can be used to 

the opportunity to record that sequence. For videotapes lO instruct viewers to view the material again, make calls or 

delivered to the viewer, a card may be delivered together other responses to the system, view or read additional, 

with the videotape. The card may contain boxes to be supporting or further material. All of this, significantly, can 

checked, blanks ti) be filled in, or choices of opaque awer- be done while maintaining complete viewer privacy. U may 

ings to be removed. Suitable instructions will be provided in also be important that some of the feedback can be local to 

the video for the viewer to scratch off or made the card in an is the viewer, while other portions of the response-strings may 

appropriate place. involve system -collection of re^nses. 

In the case of scratch-off cards, or other pre-prepared Techniques may be used for distinguishing between incor- 

cards, it is significant that colors and/or shapes might be rcct responses resulting from a failure to view or comprc- 

employed rather than known characters, or as elements in bend the programming and an error in completing a response 

known characters, like the commonly employed segments of 20 card. Techniques may also be used for identifying the most 

characters used in LED displays. Some action is required of likely error in optical character recognition review of 

the viewer to cause these characters/signs to be recorded/ responses. The sequence of numbers, letters, or other visual 

transferred to the recording area. These cards might bear or audio cues which is generated for and inserted in the 

sponsorship or advertising data and this data might also programming is also used to generate computer-readable 

generate or contain further fields, lliese cards, or on-line 25 printed information, such as bar code, corresponding 

components might cume from a third-parly source. For uniquely to that sequence. Customized codes may be devel- 

instance the on-line window might be generated for, or by, oped to correspond to cues other than alphanumeric char- 

an advertiser or vendor. A paper-based recording space acters. The printed information is applied to a return card 

which might be contained in a booklet, magazine, newspa- supplied with a videocassette on which the programming has 

per or as part of the packaging itself, may also be generated 30 been recorded. The printed information may be encoded, by 

by or for an advertiser or vendor. Such a recording space altering the order of codes or including additional codes or 

may be a peel-off videocassette label or a peel-off, tear-off, other known techniques, for security purposes. Correct 

scratch-off, or stick-on component of the videotape packag- answers lo question fields may be embedded in the bar coded 

ing or its electronic analog. This component might also string. 

contain one or more other characters, colors, icons or strings 35 Acceptable tolerances for error within which the respon- 
which might be included in the final string, or used sepa- dent will be deemed to have viewed and/or adequately 
rately. This component might also be custom-generated as comprehended the programming may be determined for 
part of the custom packaging. For instance, the sequence each position of the character string. Numerous consider- 
ABC-might already occupy a place in the sequence- alions may be u.sed in determining the.se tolerances. Exem- 
recording fields such as ABC-xxx-xxx-xxx. 'llie ABC-might 40 plary considerations are the following: (1) the first position 
be visible or might require manual intervention or problem of the string is prone to a heightened incidence of general 
solving to reveal. This problem solving might be related to user error; (2) fields which call for the answering of ques- 
the material presented on the video. For example, a scratch- tions or scratching off of a layer of obscuring material are 
off grid of multiple choice questions, or an on-line emulation subject to a heightened incidence of general user error; (3) 
of such a grid, presented by the programming or packaging 45 for positions known to include only a limited number of 
itself might yield a unique sequence of characters or signs. possible responses, statistical weighting can be employed to 
The final uses of this character string are manifold, but the assign unclear responses into one of the known responses, 
overarching intention is to provide a motivational arena for e.g., if the possible responses are to write out one of the 
the recording of this verification data. The verification data, capital letters A, B, C and D, and an OCR program reads a 
unless used in an on-line environment where the respondent 50 response as the numeral 0, the response will be assigned as 
is known to the system, usually contains fixed identifying the letter D; (4) in a system employing scratching off of 
data used to confirm the identity of either the viewer/ selected areas on the face of a response card, columns which 
respondent or the specific program viewed. It may also contain more than one removed field where one field is 
include, or consist solely of, dala strings generated in correct and not more than two fields are removed. A 
response to queries. It may further include gaming fields and 55 weighted tolerance system may also employ other strategies, 
or third -parly sponsored information. For example, a parity field may be derived from a modu lo- 
in the case of traditional videotaped or other passively cycUog of the total character string, where the base number 
consumed media, the response data can be collected swiftly of the modulus might be the number of total possible 
by 800-numbcr, or standard toll-line, call-in to an automated characters potentially employed in a single field of the 
collection system, which has been programmed in a suitable 60 string. By way of further example, a system may be used 
manner to collect response data. The response data can also which causes an known and repeatable limitation to be 
be collected by mail-in and even correlation with sealed- placed on the possible use of a given character derived from 
envelope or other hidden message systems such as peel-open the earlier or other character in the string. The source of such 
and scratch off answer fields which provide interim instruc- a check character is from within the printed information, 
tions. For viewers with suitable computer equipment, the 65 whereas ihe checked characters are from the handwritten or 
response data can be returned by electronic mail, llie receipt scratch-off fields. For example, the presence of an even 
of respon.ses is indicated by box 720. The end result though number in the first position of a string might dictate the use 
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of an odd number in the second field. Alternatively, a contrasts to traditional databases in that it mines non-linear 
template could be created which would dynamically or in a assets for presentation in a linear manner, the linear manner 
predetermined manner limit the possible characters or other not being contained within the assets. The use of a modera- 
cues which could be employed in each position of the tion layer that limits the assets responsive to the query that 
generated siring. This data can be used to optimize the OCR 5 are actuaUy presented to the user contrasts with traditional 
function of each position of the response string, as weU as database organization, m which all items responsive to a 
to dynamicaUy shift the error tolerance on a position-by- ^re preseni^l to the iLser. . . . 
oosilioD basis ® system also provides for the automatic creation of 
Bv crealing a master parity character and a known pattern new programming uniquely configured to the nsqmrements 
. / . ^- . . • . - u i_i ^ of the viewer, but which may be viewed passively. In 
in the string of characters, it ^ many Umes more probable lo ^ ^^^^ ^^^^ ^^^^^^ games, while the creation of 
that a smgle mcorrcctly read character can be isolated by the programming occuir^, the viewer must interact with 
logical cross-reference of the two strategies set forth above. uie system in order to create the programming. 
For example, if the panly check is mcorrecl by one value, gy^^^^j ^j^^ capable of arranging assets in an order, 
and the fourth position of the string should be an odd ^jjj creatic^ transitions between assets, to create a concat- 
number, bui is read as the numeral 8, it is likely that the 15 cnatcd stream of audiovisual programming that has transi- 
fourtb field contains the numeral 8 if it is also true that the xions thai appear to the viewer to have been edited by a 
fourth field cannot by design contain an even number. When human editor Even without temporal order, the assets con- 
such techniques arc combined with dynamically weighted tain sufficient transition ioformatioa, and the system con- 
general error correction, the scheme becomes tolerant of tains default transition choices, to permit assembly of pro- 
user inaccuracies and OCR errors. 20 gramming vnih fluid transitions without the intervention of 
For the presentation of \'ideo-superimposed characters, a a human editor. Numerous methods are available for order- 
static template could also be used which matches an accom- ing the assets in such a program. As discussed, assets may 
: panying printed template. Alternatively, video-generated be ordered by using a template that imposes sequential 
Q characters could be derived dynamically fmm the character requirements on the assets, an expert system, a truth table, 
Q sequence used to identify the viewer. In this scenario, 25 or a human editor may decide the order of the assets, llie 
multiple-choice questions asked uf the viewer would be attributes of the assets impose certain limitations on the 
1*^ presented on screen as would be the appropriate multiple- ordering of the assets. For example, it may not be possible 
choice answers, but the characters used to identify the for aesthetic rea.sons to place a certain clip immediately 
Q answers would be dynamically generated in response to the before or after a certain other clip. 
ilfo* viewer or programming identification sequence. For 30 The individual assets may have demographic character- 
example, if an algorithm generated the output X, then the istics. For example, an audiovisual clip may feature actors of 
\M characters A, B, C and D would be generated next to four a particular ethnic group, geographic origin, age, or other 
{] multiple-choice answers appearing on screen. If the algo- demographic characteristic. As a result, the assets may be 
[8=1 rithm generated an output of Y, the answers would be selected based on demographic search criteria. 

assigned the numerals 1,2, 3 and 4. Such a technique has the 35 The system may permit, in an on-line environment, 
f If advantage of allowing the video assets containing test mate- switching between a predetermined program and interactive 
j'lj rials to remain mutable while responding dynamically lo the video. For example, the system may provide delivery of a 
j;^ needs of the error-correcting'checking system. It will be predetermined audiovisual program so long as the viewer 
!^ understood thai the foregoing techniques may be employed does not .seek lo inleracl with the syslem, e.g., by positioning 
leal with any visual, sonic or other cue. 40 a mouse pointer on the screen and clicking. At this point, Ihe 
1^ Various points of the systems that are beheved by the system may select assets that are appropriate to the particu- 
invenlor will now be emphasized. However, the enumera- lar infonmation that the viewer appears to be seeking based 
tion of certain features believed lo be novel should not be on the information on the screen at the time. The system will 
construed as implying that other features arc not novel. then generate suitable transitions and add those assets to the 
The system provides for a database of assets which must 4S programming. For example, the program may be a tour of a 
be arranged temporally in order to be used, but which does three-dimensional model of the heart. When the user moves 
not dictate the temporal organization. Rather, the audio and a pointer or mouse to a particular portion of the screen, and 
video assets in the database are characterized by such clicks on the screen, the system may select, for example, 
parameters as content, aesthetic features, and suitability to assets incorporating more detailed information on certain 
viewers with certain demographic or psychographic charac- 50 portions of the heart corresponding to material on the 
teristics. The user can select assets through a suitable query portion of the screen where the mouse was located when 
without attention to temporal organization. The ofganization clicked. Hie system applies suitable transitions to the assets 
of assets contrasts with the organization of assets in such and incorporates these assets into the programming interac- 
fields as video games. In a video game, the permitted lively in respcm.se lo mou.se clicks or other .suitable input 
temporal orders of the assets are predetenoained by a strict 55 from the viewer. This interactive generation of programming 
branching system. Such a branching system defines pre- for passive viewing contrasts with existing systems that 
cisely which asset may b>e permitted to follow which under include a predetermined loop if the viewer does not provide 
all possible circumstances. The database of the present inputs- 
system may be termed amorphous, in contrast to the rigid The system may also be employed in an interactive 
organization of a database having a branching system. The 60 system that leads the viewer to select a desired sequence, 
general principle can be extended to other types of assets The system may, for example, provide more interesting 
that require temporal organization in order to be used. For visual data in areas of the screen that lead to the selection of 
example, a database can be constructed for use with still material that the viewer should see, or provide smooth 
photographs to be arranged into a program with an audio transitions from audiovisual assets incorporated in the pro- 
track. Interactive assets, such as two-dimensional or three- 65 gram as a result of viewer input lo audiovisual assets 
dimensional models and graphical user interfaces can also incorporated in the program to achieve a predetermined 
be catalogued in a database of this type. The present also purpose, e.g., to convey certain information to the viewer. 



